Surround view vision system that utilizes trailer camera

ABSTRACT

A vehicular surround vision system includes a plurality of vehicle cameras disposed at a vehicle and having respective fields of view exterior the vehicle, a trailer camera disposed at a rear of a trailer and having a field of view rearward of the trailer, and a display device in the vehicle. An electronic control unit (ECU) receives image data captured by the plurality of vehicle cameras. The display device displays surround view images responsive to the ECU. With the trailer hitched to the vehicle, the ECU receives image data captured by the trailer camera and the display device displays surround view images responsive to the ECU, with the displayed surround view images derived from image data captured by the trailer camera and at least some of the vehicle cameras.

CROSS REFERENCE TO RELATED APPLICATIONS

The present application claims priority of U.S. provisional applicationSer. No. 62/736,156, filed Sep. 25, 2018, and U.S. provisionalapplication Ser. No. 62/649,866, filed Mar. 29, 2018, which are herebyincorporated herein by reference in their entireties.

FIELD OF THE INVENTION

The present invention relates generally to a vehicle vision system for avehicle and, more particularly, to a vehicle vision system that utilizesmultiple cameras for a vehicle surround view system.

BACKGROUND OF THE INVENTION

Use of imaging sensors in vehicle imaging systems is common and known.Examples of such known systems are described in U.S. Pat. Nos.5,949,331; 5,670,935 and/or 5,550,677, which are hereby incorporatedherein by reference in their entireties.

SUMMARY OF THE INVENTION

The present invention provides a driver assistance system or visionsystem or imaging system for a vehicle that utilizes one or more camerasto capture image data representative of images exterior of the vehicle,and utilizes a trailer mounted camera to provide a surround view visionsystem display of the exterior regions surrounding the vehicle and thetrailer.

These and other objects, advantages, purposes and features of thepresent invention will become apparent upon review of the followingspecification in conjunction with the drawings.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a side view of a vehicle towing a trailer, showing the vehiclewith a surround view vision system and showing a wireless camerainstalled at the rear of the trailer and connected to the surround viewsystem via a wireless link;

FIG. 2 is a system block diagram of the surround view vision system ofFIG. 1;

FIG. 3 is a side view of another vehicle towing a trailer, showing thevehicle with a surround view vision system and showing a camerainstalled at the rear of the trailer with the compressed videocommunicated to the vehicle ECU via a power line of the trailer;

FIG. 4 is a system block diagram of the surround view vision system ofFIG. 3; and

FIG. 5 is a system block diagram of the surround view vision system,showing integration of the wireless camera subsystem into an existingHead unit/SVS unit.

LEGEND

-   -   100—Trailer camera    -   102—Vehicle towing the trailer    -   104—Trailer equipped with camera    -   106—Antenna mounted on the vehicle    -   108 a, b . . . n—Surround view cameras    -   108 a—Backup camera    -   110—Trailer camera antenna    -   112—Head unit    -   114—Surround View System ECU    -   116—Existing power line in the trailer    -   118—Trailer powerline module    -   120—Vehicle powerline module    -   121—LVDS splitter    -   123—Serializer    -   125—Wireless camera extension    -   124—SOC (system-on-chip)/Processor    -   122—Wireless chipset/Module    -   126—External wireless antenna    -   128—Vehicle power source

DESCRIPTION OF THE PREFERRED EMBODIMENTS

A vehicle vision system and/or driver assist system and/or objectdetection system and/or alert system operates to capture images exteriorof the vehicle and may process the captured image data to display imagesand to detect objects at or near the vehicle and in the predicted pathof the vehicle, such as to assist a driver of the vehicle in maneuveringthe vehicle in a rearward direction. The vision system includes an imageprocessor or image processing system that is operable to receive imagedata from one or more cameras and provide an output to a display devicefor displaying images representative of the captured image data.Optionally, the vision system may provide display, such as a rearviewdisplay or a top down or bird's eye or surround view display or thelike.

A surround view vision system for a vehicle 102 includes an electroniccontrol unit (ECU) 114 and a display or head unit 112 at the vehicle.The system includes a plurality of vehicle-mounted cameras 108 a, 108 b. . . 108 n (FIG. 2), which capture image data representative of thearea exterior of and surrounding the vehicle and within the respectivefields of views of the cameras. Image data captured by the cameras iscommunicated to the ECU 114 a, whereby video images derived from thecaptured image data are displayed at the display device or head unit 112for viewing by a driver of the vehicle while maneuvering the vehicle.

As shown in FIG. 1, the vehicle 102 may tow a trailer 104, with a camera100 mounted on the trailer 104. The camera 100 transmits compressedvideo data, via an antenna 110 mounted on the trailer 104, to an antenna106 mounted on the vehicle 102. The antenna 106 is connected to thesurround view system ECU 114 (such as via a vehicle communication bus ornetwork or the like). The surround view system ECU 114 processes theimage data received via the wireless link and displays the video imageson the head unit 112. The displayed video images comprise images derivedfrom image data captured by the front camera 108 a and side cameras (andoptionally the rear camera 108 b) of the vehicle and image data capturedby the trailer camera so as to provide a surround view display of thearea around the vehicle and trailer.

Optionally, and such as shown in FIGS. 3 and 4, the compressed videofrom the camera 100 mounted on the trailer 104 may be transmitted via anexisting powerline 116 of the trailer, utilizing a communication overpowerline module 118, and the transmitted data is received by thesurround view system ECU 114 at the vehicle 102 via a communication overpowerline module 120 installed at the vehicle 102. The module 120includes EMI suppression that limits or prevents the interferencegenerated due to communication over powerline to enter the vehicle powerline. The electrical connection of the trailer powerline 116 and thevehicle powerline module 120 may be made at a trailer and vehicleharness connection when the trailer is hitched to the vehicle.

Optionally, and such as shown in FIG. 5, a wireless extension module 125may be integrated with the head unit 112 or the surround view system114. The video data is received from the wireless camera 100 via anexternal wireless antenna 126 and wireless chipset/module 122, and thecompressed video data is decompressed using the processor 124 andconverted to low-voltage differential signaling (LVDS) via theserializer 123 and then combined with the other video stream from thebackup camera 108 a (at the LVDS splitter 121) and transmitted to theSVS (surround view system or surround vision system) unit 114 or headunit 112 via the LVDS link 118. The control commands from the head unit112 are transmitted back to the wireless camera unit 100 via backchannel communication using the LVDS link 118.

When no trailer or trailer camera is connected to the system, the ECUprocesses image data captured by the vehicle cameras (such as a forwardcamera having a forward field of view, a rearward camera having arearward field of view, a driver-side camera having a sideward field ofview at the driver-side of the vehicle, and a passenger-side camerahaving a sideward field of view at the passenger-side of the vehicle) togenerate the surround view video images for display at the displaydevice. When a trailer is hitched to the vehicle and the trailer camerais in communication with the ECU, the ECU processes image data capturedby the trailer camera and by the vehicle cameras (such as a forwardcamera having a forward field of view, a driver-side camera having asideward field of view at the driver-side of the vehicle, and apassenger-side camera having a sideward field of view at thepassenger-side of the vehicle) to generate the surround view videoimages for display at the display device. When displaying the surroundview of the vehicle and trailer, the ECU may or may not process imagedata captured by a rearward vehicle camera having a rearward field ofview.

Thus, the present invention provides a method and apparatus forstreaming video image data from a wireless camera mounted on a trailerand a display on the vehicle head unit (of the vehicle towing thetrailer). The wireless camera mounted on the trailer transmitscompressed video data via proprietary wireless protocol or standardWi-Fi protocol to the surround view system installed in the vehicle. Thesurround view system display that is connected to the head unit displaysthe streamed video on the head unit. Thus, there is no dependency on thehead unit wireless protocol support and the system can utilizeproprietary wireless protocols depending on the application need.

Optionally, a plurality of trailer cameras may be disposed at thetrailer, the plurality of trailer cameras at least comprising (i) a reartrailer camera disposed at a rear portion of the trailer and having afield of view at least rearward of the trailer, (ii) a left-side trailercamera disposed at a left side of the trailer and having a field of viewat least sideward of the trailer and (iii) a right-side trailer cameradisposed at a right side of the trailer and having a field of view atleast sideward of the trailer. Image data captured by the trailercameras is provided to the ECU and the display may display video imagesderived from the captured image data provided to the ECU. The systemthus may provide surround view video images at the sides and rear of thetrailer.

Optionally, the driver may select the images for display, such as viaactuation of a user input or the like in the vehicle, whereby thedisplay screen may display video images derived from image data capturedby the rear trailer camera or video images derived from image datacaptured by the plurality of trailer cameras, depending on the desiredview during the driving maneuver of the vehicle and trailer, and thedriver may make a selection such that video images derived from imagedata captured by the vehicle cameras are not displayed along with thedisplayed video images derived from image data captured by the selectedtrailer camera or cameras. Optionally, the trailer may only have arear-mounted and rearward viewing trailer camera, whereby the regions atthe sides of the trailer may be encompassed by the respective fields ofview of side-mounted vehicle cameras.

Optionally, the camera or cameras mounted on the trailer may transmitthe video image data to the surround view system ECU via an existing DCpower-over-line that connects to the back of the trailer. The advantageof this approach is that multiple cameras may be installed inside oroutside of the trailer and all the video image data could be collectedby the demodulator/decoder module that is installed in the vehicle. Thisis similar to the power line communication method applied to thecameras. When the camera needs to be installed inside the trailer, thenthe camera is connected to the power line and the communication link isestablished between the camera and the decoder in the vehicle and thecompressed video data is transmitted to the surround view system ECU.Video images representative of the scene inside the trailer and derivedfrom image data captured by the inside camera may be displayed in thevehicle for viewing by a driver or occupant of the vehicle. The insidetrailer camera may operate to capture image data responsive to a userinput or selection, whereby the display displays video imagesrepresentative of the image data captured by the inside trailer camerafor viewing by the driver or an occupant of the vehicle. When the systemis displaying the video images representative of the image data capturedby the inside trailer camera, the display does not display video imagesrepresentative of image data captured by the other cameras.

Optionally, the wireless extension module may be added to the existingsurround view system that could extend the capability of the system toconnect the trailer wireless camera unit and to stream the video imagedata to the head unit. Optionally, if the vehicle does not have asurround view system ECU, then the wireless extension module could beadded in between the backup camera and the head unit LVDS link. In bothcases, a software update may be made to the surround view system as wellas to the head unit to control the wireless camera.

The vision system and/or trailer camera or cameras may utilize aspectsof the systems and cameras described in U.S. patent application Ser. No.16/364,245, filed Mar. 26, 2019 (Attorney Docket MAG04 P3562), Ser. No.16/250,503, filed Jan. 17, 2019 (Attorney Docket MAG04 P-3521), and/orSer. No. 16/250,480, filed Jan. 17, 2019 (Attorney Docket MAG04 P-3520),which are hereby incorporated herein by reference in their entireties.

The vision system and/or processing and/or camera and/or circuitry mayutilize aspects described in U.S. Pat. Nos. 9,233,641; 9,146,898;9,174,574; 9,090,234; 9,077,098; 8,818,042; 8,886,401; 9,077,962;9,068,390; 9,140,789; 9,092,986; 9,205,776; 8,917,169; 8,694,224;7,005,974; 5,760,962; 5,877,897; 5,796,094; 5,949,331; 6,222,447;6,302,545; 6,396,397; 6,498,620; 6,523,964; 6,611,202; 6,201,642;6,690,268; 6,717,610; 6,757,109; 6,802,617; 6,806,452; 6,822,563;6,891,563; 6,946,978; 7,859,565; 5,550,677; 5,670,935; 6,636,258;7,145,519; 7,161,616; 7,230,640; 7,248,283; 7,295,229; 7,301,466;7,592,928; 7,881,496; 7,720,580; 7,038,577; 6,882,287; 5,929,786 and/or5,786,772, and/or U.S. Publication Nos. US-2014-0340510;US-2014-0313339; US-2014-0347486; US-2014-0320658; US-2014-0336876;US-2014-0307095; US-2014-0327774; US-2014-0327772; US-2014-0320636;US-2014-0293057; US-2014-0309884; US-2014-0226012; US-2014-0293042;US-2014-0218535; US-2014-0218535; US-2014-0247354; US-2014-0247355;US-2014-0247352; US-2014-0232869; US-2014-0211009; US-2014-0160276;US-2014-0168437; US-2014-0168415; US-2014-0160291; US-2014-0152825;US-2014-0139676; US-2014-0138140; US-2014-0104426; US-2014-0098229;US-2014-0085472; US-2014-0067206; US-2014-0049646; US-2014-0052340;US-2014-0025240; US-2014-0028852; US-2014-005907; US-2013-0314503;US-2013-0298866; US-2013-0222593; US-2013-0300869; US-2013-0278769;US-2013-0258077; US-2013-0258077; US-2013-0242099; US-2013-0215271;US-2013-0141578 and/or US-2013-0002873, which are all herebyincorporated herein by reference in their entireties.

Optionally, the display for displaying images captured by one or more ofthe imaging sensors for viewing by the driver of the vehicle while thedriver is normally operating the vehicle, may, for example, the visionsystem may include a video display device, such as by utilizing aspectsof the video display systems described in U.S. Pat. Nos. 5,530,240;6,329,925; 7,855,755; 7,626,749; 7,581,859; 7,446,650; 7,338,177;7,274,501; 7,255,451; 7,195,381; 7,184,190; 5,668,663; 5,724,187;6,690,268; 7,370,983; 7,329,013; 7,308,341; 7,289,037; 7,249,860;7,004,593; 4,546,551; 5,699,044; 4,953,305; 5,576,687; 5,632,092;5,708,410; 5,737,226; 5,802,727; 5,878,370; 6,087,953; 6,173,501;6,222,460; 6,513,252 and/or 6,642,851, and/or U.S. Publication Nos.US-2014-0022390; US-2012-0162427; US-2006-0050018 and/orUS-2006-0061008, and/or International Publication Nos. WO 2010/099416;WO 2011/028686; WO 2012/075250; WO 2013/019795; WO 2012/075250; WO2012/145822; WO 2013/081985; WO 2013/086249 and/or WO 2013/109869,and/or U.S. Publication No. US-2012-0162427, which are herebyincorporated herein by reference in their entireties.

Changes and modifications in the specifically described embodiments canbe carried out without departing from the principles of the invention,which is intended to be limited only by the scope of the appendedclaims, as interpreted according to the principles of patent lawincluding the doctrine of equivalents.

1. A vehicular surround vision system, said vehicular surround visionsystem comprising: a plurality of vehicle cameras disposed at a vehicleand having respective fields of view exterior the vehicle; a trailercamera disposed at a rear of a trailer and having a field of view atleast rearward of the trailer; a display device in the vehicle; anelectronic control unit (ECU) in the vehicle; wherein, with the trailernot hitched to the vehicle and during a driving maneuver of the vehicle,(i) image data captured by said plurality of vehicle cameras is providedto said ECU and (ii) said display device displays surround view videoimages derived from image data captured by said plurality of vehiclecameras and provided to said ECU; and wherein, with the trailer hitchedto the vehicle and during a driving maneuver of the vehicle and trailer,(i) image data captured by said trailer camera and by said plurality ofvehicle cameras is provided to said ECU and (ii) said display devicedisplays surround view video images derived from (a) image data capturedby said trailer camera and provided to said ECU and (b) image datacaptured by at least some of said plurality of vehicle cameras andprovided to said ECU.
 2. The vehicular surround vision system of claim1, wherein said trailer camera is one of a plurality of trailer camerasdisposed at the trailer, and wherein said plurality of trailer camerasat least comprises (i) a left-side trailer camera disposed at a leftside of the trailer and having a field of view at least sideward of thetrailer and (ii) a right-side trailer camera disposed at a right side ofthe trailer and having a field of view at least sideward of the trailer.3. The vehicular surround vision system of claim 2, wherein, with thetrailer hitched to the vehicle and during a driving maneuver of thevehicle and trailer, (i) image data captured by said plurality oftrailer cameras is provided to said ECU and (ii) said display devicedisplays surround view video images derived from image data captured bysaid plurality of trailer cameras and provided to said ECU.
 4. Thevehicular surround vision system of claim 1, wherein display by saiddisplay device of video images derived from image data captured by saidtrailer camera is responsive at least in part to a user input.
 5. Thevehicular surround vision system of claim 4, wherein the user input isactuated to selectively display by said display device (i) video imagesderived from image data captured by said trailer camera and provided tosaid ECU or (ii) surround view video images derived from (a) image datacaptured by said trailer camera and provided to said ECU and (b) imagedata captured by at least some of said plurality of vehicle cameras andprovided to said ECU.
 6. The vehicular surround vision system of claim1, comprising a transmitting antenna disposed at the trailer, wherein,with the trailer hitched to the vehicle, said transmitting antennawirelessly transmits image data captured by said trailer camera to adata receiving device disposed at the vehicle, and wherein said datareceiving device provides the transmitted captured image data to saidECU.
 7. The vehicular surround vision system of claim 6, wherein saiddata receiving device is connected to a communication link between saidECU and a rear backup camera of said plurality of vehicle cameras. 8.The vehicular surround vision system of claim 7, wherein said datareceiving device comprises (i) a processor that decompresses compressedvideo image data and (ii) a serializer that converts the decompressedvideo image data to low-voltage differential signaling.
 9. The vehicularsurround vision system of claim 1, comprising a receiving antenna at thevehicle that receives image data captured by said plurality of vehiclecameras and said trailer camera.
 10. The vehicular surround visionsystem of claim 9, wherein said receiving antenna communicates thereceived captured image data to said ECU.
 11. The vehicular surroundvision system of claim 10, wherein said ECU decompresses thecommunicated received captured image data.
 12. The vehicular surroundvision system of claim 1, comprising a powerline at the trailer, whereinthe powerline is electrically connected to a wire harness of the vehiclewhen the trailer is hitched to the vehicle, and wherein the powerlinecommunicates image data captured by said trailer camera to a powerlinemodule disposed at the vehicle, and wherein said powerline modulereceives the communicated captured image data and provides the receivedcommunicated captured image data to said ECU.
 13. The vehicular surroundvision system of claim 1, comprising an interior trailer camera disposedin the trailer and having a field of view interior the trailer, wherein,with the trailer hitched to the vehicle, image data captured by saidinterior trailer camera is provided to said ECU, and wherein saiddisplay device displays video images that are representative of imagedata captured by said interior trailer camera and provided to said ECU.14. A vehicular vision system, said vehicular vision system comprising:at least one vehicle camera disposed at a vehicle and having a field ofview exterior the vehicle; a trailer camera disposed at a rear of atrailer and having a field of view at least rearward of the trailer; adisplay device in the vehicle; an electronic control unit (ECU) in thevehicle; wherein, with the trailer not hitched to the vehicle and duringa driving maneuver of the vehicle, (i) image data captured by said atleast one vehicle camera is provided to said ECU and (ii) said displaydevice displays video images derived from the image data captured bysaid at least one vehicle camera and provided to said ECU; and wherein,with the trailer hitched to the vehicle and during a driving maneuver ofthe vehicle, (i) image data captured by said trailer camera and said atleast one vehicle camera is provided to said ECU and (ii) said displaydevice displays video images derived from at least one selected from thegroup consisting of (a) image data captured by said trailer camera andprovided to said ECU and (b) image data captured by said at least onevehicle camera and provided to said ECU.
 15. The vehicular vision systemof claim 14, comprising an interior trailer camera disposed in thetrailer and having a field of view interior the trailer, wherein, withthe trailer hitched to the vehicle, image data captured by said interiortrailer camera is provided to said ECU, and wherein said display devicedisplays video images that are representative of image data captured bysaid interior trailer camera and provided to said ECU.
 16. The vehicularvision system of claim 15, wherein said display device displays videoimages that are representative of image data captured by said interiortrailer camera responsive to a user input.
 17. The vehicular visionsystem of claim 14, wherein display by said display device of videoimages derived from image data captured by said trailer camera isresponsive at least in part to a user input.
 18. The vehicular visionsystem of claim 17, wherein the user input is actuated to selectivelydisplay by said display device (i) video images derived from image datacaptured by said trailer camera and provided to said ECU or (ii) videoimages derived from (a) image data captured by said trailer camera andprovided to said ECU and (b) image data captured by said at least onevehicle camera and provided to said ECU.
 19. A vehicular surround visionsystem, said vehicular surround vision system comprising: a plurality ofvehicle cameras disposed at a vehicle and having respective fields ofview exterior the vehicle, wherein said plurality of vehicle cameras atleast comprises (i) a rear vehicle camera disposed at a rear of thevehicle and having a field of view at least rearward of the vehicle,(ii) a left-side vehicle camera disposed at a left side of the vehicleand having a field of view at least sideward of the vehicle and (iii) aright-side vehicle camera disposed at a right side of the vehicle andhaving a field of view at least sideward of the vehicle; a plurality oftrailer cameras disposed at a trailer, wherein said plurality of trailercameras at least comprises (i) a rear trailer camera disposed at a rearof the trailer and having a field of view at least rearward of thetrailer, (ii) a left-side trailer camera disposed at a left side of thetrailer and having a field of view at least sideward of the trailer and(iii) a right-side trailer camera disposed at a right side of thetrailer and having a field of view at least sideward of the trailer; adisplay device in the vehicle; an electronic control unit (ECU) in thevehicle; wherein, with the trailer not hitched to the vehicle and duringa driving maneuver of the vehicle, (i) image data captured by saidplurality of vehicle cameras is provided to said ECU and (ii) saiddisplay device displays surround view video images derived from imagedata captured by said plurality of vehicle cameras and provided to saidECU; wherein, with the trailer hitched to the vehicle and during adriving maneuver of the vehicle and trailer, (i) image data captured bysaid plurality of trailer cameras and by said plurality of vehiclecameras is provided to said ECU and (ii) said display device displayssurround view video images derived from one selected from the groupconsisting of (a) image data captured by said plurality of trailercameras and provided to said ECU and (b) image data captured by saidplurality of trailer cameras and by at least some of said plurality ofvehicle cameras and provided to said ECU; and wherein selection of saidone selected from the group consisting of (a) image data captured bysaid plurality of trailer cameras and provided to said ECU and (b) imagedata captured by said plurality of trailer cameras and by at least someof said plurality of vehicle cameras and provided to said ECU is madevia actuation of a user input by a driver of the vehicle.
 20. Thevehicular surround vision system of claim 19, comprising an interiortrailer camera disposed in the trailer and having a field of viewinterior the trailer, wherein, with the trailer hitched to the vehicle,image data captured by said interior trailer camera is provided to saidECU, and wherein, responsive to actuation of another user input by thedriver of the vehicle, said display device displays video images thatare representative of image data captured by said interior trailercamera and provided to said ECU.